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ABSTRACT 

Vehicle  survivability  in  the  form  of  anti-tampering  tools  is  an  important 
part  of  the  ECS  community  of  vehicles.  Autonomous  or  semi- 
autonomous  vehicles  traversing  through  unknown  terrain  require  the 
ability  to  detect,  predict,  and  avoid  potential  aggression  from  hostile 
forces.  Expanding  upon  pedestrian  detection  techniques  will  provide  a 
viable  solution  in  the  near  term. 

With  the  current  vision  based  pedestrian  detection  algorithms,  an  anti¬ 
tampering  suite  can  be  developed.  First,  the  system  must  determine 
people  in  the  scene.  Once  this  is  accomplished,  the  people  can  be 
checked  for  weapons  and  their  movements  can  be  tracked.  Finally 
deciphering  the  tracked  movements  can  help  distinguish  between 
friendly  and  unfriendly  actions  by  people. 

This  paper  will  discuss  the  need  for  an  anti-tampering  suite  and  how 
pedestrian  detection  techniques  can  be  used  to  address  this  need. 


1.  Introduction 

The  term  anti-tamper  when  related  to  security  systems  means  either  a  security  device  that 
destroys  crucial  data  just  before  the  information  falls  into  enemy  hands  or  a  system  that 
has  the  ability  to  detect  when  possible  tampering  of  a  system  might  take  place.  These 
capabilities  needed  throughout  the  army  and  the  other  forces.  This  paper  will  only  talk 
about  the  latter;  more  specifically,  detection  of  a  person  that  has  hostile  intent  towards  the 
robotic  system. 

The  first  section  will  talk  about  the  need  for  a  good  pedestrian  detection  system  as  a 
baseline  for  any  anti-tamper  algorithms.  The  lower  quality  the  human  detector,  the  lower 
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quality  the  anti-tamper  system  will  be.  The  following  section  will  discuss  the  efforts  that 
the  Intelligence  Systems  Division  is  focusing  on  with  respect  to  pedestrian  detection. 
Section  four  will  discuss  anti-tamper  systems  available,  and  section  five  will  talk  about 
the  current  work  that  Intelligence  Systems  is  focusing  on  with  respect  to  anti-tamper. 
Finally,  section  six  will  conclude  the  paper  followed  by  the  references  (section  seven). 


2.  Pedestrian  Detection  Systems 

Pedestrian  detection  is  an  important  field  of  research.  Autonomous  and  semi- 
autonomous  vehicles  need  to  identify  people  while  traversing  through  the  terrain  in  order 
to  take  appropriate  actions  to  avoid  them.  Driver  awareness  systems,  like  those  proposed 
by  the  Department  of  Transportation  Intelligent  Transportation  Systems  Division,  need 
the  ability  to  alert  drivers  of  potential  problems  when  driving  through  urban  areas. 
Additionally,  the  Department  of  Defense  needs  pedestrian  detection  for  path  following 
and  mule  operations  on  robotic  vehicles.  In  this  paper,  the  focus  is  on  pedestrian 
detection  and  its  use  in  anti-tamper  systems  for  robotic  vehicles. 

The  research  area  of  pedestrian  detection  is  very  large.  There  are  many  different 
approaches  to  this  problem.  Some  use  LADAR  or  laser  scanners  to  retrieve  a  3D  map  of 
the  terrain  and  detect  pedestrians  [1,2, 3, 4],  another  uses  ultrasonic  sensors  to  determine 
the  reflection  of  pedestrians  [5].  Radar  is  also  popular  for  detecting  pedestrians  similar  to 
ultrasonic  sensors;  by  measuring  the  reflection  of  possible  targets  and  determining  if  they 
are  pedestrians  or  not  [6,7]. 

A  natural  choice  for  a  sensor  is  vision  because  it  is  based  on  how  people  perceive 
pedestrians.  Within  the  area  of  computer  vision  there  are  infrared  vision,  monocular 
vision,  and  stereo  vision  processes.  Each  vision  system  has  its  own  advantages  and 
disadvantages.  Infrared  systems  [8,9,10,11]  are  not  as  sensitive  to  lighting  conditions 
when  compared  to  other  visual  sensors.  However,  they  are  more  expensive  and  the 
image  quality  is  not  as  good.  Monocular  vision  systems  [12,13,21]  are  cheap  and  require 
lower  processing  power,  but  they  perform  poorly  at  providing  range  data  and  are  more 
sensitive  to  color  and  lighting.  Stereo  vision  systems  [14,15,16,17,18,19,20]  have  the 
advantage  of  being  able  to  view  potential  objects  from  two  points  of  view.  They  are  also 
used  to  detect  dispersion  (or  depth)  of  objects.  The  drawbacks  of  stereo  vision  systems 
are  that  they  require  more  processing  time  and  are  sensitive  to  color  and  lighting 
conditions. 

Vision  Systems:  Pedestrian  detection  is  a  difficult  problem  because  of  processing  speed, 
robustness  of  vision  sensor’s  algorithms,  and  a  lack  of  maturity  in  computational 
intelligent  systems  to  recognize  everyday  object.  For  the  Department  of  Defense, 
specifically  TARDEC,  vision  based  detection  is  important  for  non-evasive  pedestrian 
detection  systems  in  path  following,  mule  operations,  surveillance,  and  driver  awareness. 
The  problem  increases  in  difficulty  when  considering  the  movement  of  the  sensors, 
uncontrolled  outdoor  environments,  and  variations  in  pedestrian’s  appearance  and  pose. 
There  are  many  types  of  algorithms  that  try  to  address  these  problems. 
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Motion  based  systems  are  used  to  detect  pedestrians  from  image  sequences.  They  take 
into  account  the  temporal  information  to  detect  periodic  features  of  human  movement 
[15,17,18].  This  technique  reduces  the  number  of  false  positives  from  other  methods,  but 
requires  a  lead  time  of  images,  which  causes  a  delay  in  the  detection.  Another  drawback 
is  that  it  is  unable  to  detect  people  standing  or  making  unusual  movements  (like 
jumping). 

Template  based  systems  [14,16,20]  can  be  used  on  single  frames  so  they  do  not  require  a 
lead  time  or  movement  of  a  pedestrian.  The  problem  with  most  of  these  procedures  is 
that  they  have  difficulty  in  detecting  variations  in  a  pedestrian’s  appearance  or  pose. 

One  other  common  method  is  to  detect  body  parts  of  person,  then  put  them  together  in  a 
logical  form  to  determine  the  confidence  of  the  target  being  a  person  or  not  [19,21].  This 
requires  a  lot  of  processing,  but  is  good  at  detecting  occluded  people.  Many  false 
positives  are  due  to  matching  potential  body  parts  of  things  unrelated. 


3.  Intelligent  System’s  Pedestrian  Detection  Program 

The  pedestrian  detection  program  in  the  Intelligent  Systems  division  looks  at  several 
different  types  of  algorithms  to  detect  people.  The  main  focus  is  on  visual  and  infrared 
sensor  based  pedestrian  detection  systems.  Each  algorithm  is  modeled  for  the 
environment  it  will  be  used  in,  from  driver  awareness  systems  to  pedestrian  following 
autonomous  systems.  Below  is  a  description  of  the  types  of  pedestrian  detection  projects 
currently  being  researched  by  the  Human  Detection  and  Intent  Analysis  Lab  (HDIAL), 
under  the  Intelligent  Systems  Division. 


Figure  1.  Gray-Scale  Pedestrian  Detection 
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Human  Localization  Using  Gray  Scale  Stereo  Imagery.  This  programs  main  purpose  is 
two-fold.  First  it  is  used  to  detect  humans  in  a  single,  stereo  image  and  alert  the  driver  of 
their  location;  second  it  provides  an  autonomous  or  semi-autonomous  robotic  system  the 
location  of  people  in  the  scene.  It  uses  gray  scale  intensity  mapping  with  depth 
information  from  the  stereo  cameras  to  single  out  possible  people.  Then  it  removes  most 
all  false  positives  by  doing  a  head  check  of  the  candidates.  Finally,  it  sends  (or  displays) 
the  pedestrian  location.  See  figure  1  for  a  visual  view  of  this. 

Human  Localization  Using  Infrared  Stereo  Imagery.  This  program  is  used  in  conjunction 
with  the  Human  Lx)calization  using  gray  scale  imagery  to  improve  the  performance  of 
pedestrian  detection  from  single  images.  It  first  views  the  higher  intensity  areas  and 
computes  the  distance  of  the  areas  in  both  left  and  right  camera  views.  Any  non-matched 
items  are  removed.  Then  it  populates  left  over  regions  based  on  distance  from  the  camera 
and  combines  them  together  based  on  proximity  and  typical  human  length/width  ratios. 
The  final  processing  checks  for  a  head  in  the  regions  of  interest  and  removes  candidates 
without  one.  See  figure  2  for  a  visual  view  of  this  processes. 
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Figure  2.  Infrared  Pedestrian  Detection  Process. 


The  human  localization  project  will  combine  both  processing  results  of  the  infrared 
algorithms  and  the  gray  scale  algorithms  in  a  dynamic  intelligent  algorithm.  The  intent  is 
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to  choose  the  best  pieces  of  information  from  both  gray  scale  and  infrared  processing 
algorithms  based  on  the  vehicles  current  environmental  conditions. 

Color  Stereo  Pedestrian  Detection.  This  project’s  main  goal  is  to  increase  the  current 
pedestrian  detection  systems  by  adding  intelligent  techniques  to  color  processing.  The 
color  image  is  processed  to  cluster  different  shades  of  colors  and  distances  from  the 
camera.  The  image  is  then  matched  against  templates  for  body  parts  (arms,  legs,  torsos, 
heads,  etc.).  Each  possible  body  part  is  identified  and  a  location  from  each  other  is  used 
to  determine  if  it  is  a  feasible  person  (as  well  as  which  person  each  part  belongs  to).  This 
will  eliminate  problems  with  occlusion  of  people  in  a  scene. 

Pedestrian  Following.  The  main  purpose  of  this  project  is  to  create  an  algorithm  to  track 
a  particular  person  using  color  stereo  cameras.  It  will  be  implemented  on  several 
different  robotics  platforms.  The  system  will  be  operator  initiated  through  the  selection 
of  a  specific  person  to  follow  by  clicking  on  the  person  through  the  human  robot  interface 
(from  an  image  provided  by  one  of  the  two  cameras  attached  to  the  robot).  Next,  the 
pedestrian  is  segmented  and  blob  clustering  is  performed  based  on  color  and  disparity. 
This  processed  image  is  used  as  a  template  for  the  next  frame.  As  the  person  starts  to 
move,  the  region  of  movement  from  one  frame  to  the  next  is  calculated  and  the 
segmented  image  from  the  previous  frame  is  used  as  a  template  to  find  the  location  of  the 
pedestrian  and  matched.  A  distance  from  the  cameras  (0,0,0  world  coordinates)  to  the 
pedestrian  (x,y,z  world  coordinates)  is  computed  and  sent  to  the  mobility  process  of  the 
robot.  The  template  matching  is  dynamic  since  each  template  changes  from  each  frame. 
The  computed  location  can  be  updated  every  second  or  every  minute,  based  on  the  type 
of  following  preferred.  It  will  also  be  designed  to  work  on  any  GPS  waypoint  robotic 
vehicle.  The  waypoints  will  be  computed  by  the  calculations  of  the  pedestrian  locations. 

Fused  Infrared  and  Gray  Scale  Pedestrian  Detection/Enhancement.  This  project  is  a  joint 
effort  between  the  HDIAL  and  the  Perception  Lab.  The  Perception  Lab’s  effort  is 
focused  on  human  perception  studies  on  fusion  techniques  between  gray  scale  and 
infrared  imagery.  The  HDIAL’s  focus  is  on  using  the  same  fusion  techniques  but 
applying  machine  intelligence  to  the  problem.  The  goal  of  this  project  is  to  have  the 
computer  detect  pedestrians  with  comparable  results  record  by  the  human  studies.  An 
investigation  into  computer  enhancement  of  the  fused  imagery  will  also  be  performed. 


4.  Anti-Tamper  Systems 

Anti-tamper  systems  are  important  to  autonomous  systems  due  to  the  need  to  characterize 
a  threat  from  potential  robotic  or  human  interactions.  The  ideal  anti-tamper  system 
would  be  one  that  can  see  a  person  before  being  seen,  move  to  a  tactical  position  out  of 
view,  and  take  appropriate  actions  once  being  seen  based  on  the  person’s  perceived 
actions.  It  would  be  able  to  segment  out  the  human,  determine  the  human’s  intent,  and 
either  implement  a  warning  system  to  deter  the  human,  move  out  of  the  danger  zone  from 
the  human,  or  radio  an  operator  for  further  instructions. 
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Today’s  anti-tamper  systems  are  mainly  based  on  security  devices  where  the  sensors  are 
in  a  static  position,  like  building  and  parking  lot  security.  For  the  army,  specifically, 
military  ground  robotics,  the  need  is  to  put  these  technologies  on  to  moving  platforms.  A 
system  that  not  only  detects  people,  but  also  has  the  ability  to  determine  the  intent  of  the 
person  requires  an  intelligence  algorithm  not  available  on  most  anti-tamper  type  systems. 
An  anti-tamper  system  can  be  classified  into  two  types  of  systems:  a  dumb  system  and  a 
smart  system. 

A  dumb  anti-tamper  systems  main  object  is  to  identify  moving  objects.  Many  use  radar, 
sonar,  or  even  vision  as  the  main  sensor  to  identify  changes  in  scenery,  like  movement  of 
a  person  or  animal.  Once  movement  is  identified  from  the  system,  either  a  warning 
signal  to  deter  the  moving  object  or  an  uplink  to  an  operator  to  determine  actions  to  take 
is  initiated.  These  types  of  anti-tamper,  or  security,  systems  are  in  the  “dumb  anti¬ 
tamper”  class  because  no  intelligence  is  modeled  to  determine  intent  of  the  moving 
object. 

Smart  anti-tamper  systems  mainly  consist  of  a  sensor,  or  multiple  sensors,  and  an 
intelligent  algorithm  that  determines  not  only  motion  in  the  sensor  field  of  view,  but  also 
reasoning  on  the  changes.  Because  of  the  nature  of  determining  intent  in  the  scene,  the 
system  needs  the  details  offered  by  visual  systems.  Some  anti-tamper  systems  may 
include  other  sensors,  but  the  main  focus  of  the  intent  data  is  based  on  vision. 

There  are  very  few  smart  anti-tamper  systems  that  are  being  used  (or  soon  to  be  used)  on 
security  systems  that  determine  the  intent  of  a  person  [22,23]  or  a  scene.  The  maturity  of 
the  smart  systems  have  not  reached  a  usable  level,  but  the  research  into  accomplishing 
this  task  is  increasing.  The  main  focus  on  this  research  is  either  through  rule/logic  based 
systems  [24,25,26]  or  through  probabilistic  networks  [27,28]. 


5.  Current  Anti-Tamper  Program 

The  anti-tamper  program  in  the  intelligent  systems  division  split  the  problem  up  into  two 
parts:  finding  the  humans  (or  pedestrian  detection)  and  determining  the  intent  of  the 
humans.  Pedestrian  detection  is  a  hard  problem  on  its  own.  However,  great  strides  are 
being  made  to  isolate  a  person  from  the  scenario.  The  HDIAL  has  designed  and  built 
many  systems  using  their  algorithms  discussed  above.  Results  are  good.  High  detection 
rates  with  minimal  false  positives. 

To  determine  intent,  designs  are  being  tested  as  well  as  software  that  is  currently 
available.  Research  in  this  area  is  continuing  and  a  system  design  should  be  completed 
by  September  2005.  Testing  of  the  anti-tamper  system  should  take  place  in  June  2006. 


6.  Conclusion 

Within  the  next  decade  robotic  vehicles  will  be  introduced  into  the  battlefield  in  large 
numbers  as  a  result  of  FCS.  This  will  dictate  a  change  in  doctrine  on  how  the  Army 
fights.  Robots  and  soldiers  will  be  in  the  field  together  and  need  to  coexist  and  function 
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in  teams.  It  is  imperative  that  these  systems  are  free  from  aggressive  actions  by  opposing 
forces.  This  is  accomplished  by  including  an  anti-tamper  systems  module  to  all 
battlefield  robotic  systems. 

TARDEC’s  Intelligent  System’s  Human  Detection  and  Intent  Analysis  Lab  is  focused  on 
providing  a  quality  anti-tamper  system  to  detect  aggressive  actions  from  humans  and 
supply  information  to  operators  when  hostile  actions  have  been  identified.  This  effort 
will  feed  Intelligence  System’s  Army  Technology  Objectives  (ATOs)  such  as  Armed 
Robotic  Vehicle  (ARV)  Robotic  Technologies  (ART)  and  Human  Robot  Interface  (HRI). 
These  ATOs  provide  technologies  that  transition  to  FCS  platforms. 
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